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Method and apparatus for representing a predeterminable 

region in multidimensional data sets 

The invention relates to a method of representing a 
predeterminable region in multidimensional data sets 
according to the preamble of claim 1 and corresponding 
apparatus for carrying out such a method according to the 
preamble of claim 8. 

Methods and apparatus of this type are known for example 
from WO 98/20360 of from PCT/EP03/01192 . In this case, 
multidimensional data sets are generated in order to image 
multidimensional objects, such as for example the heart or 
liver of a human, the surface of a semiconductor or the weld 
seam of a packaging. Such multidimensional data sets are 
generated by recording the object by a suitable recording 
process, such as for example an x-ray image, an ultrasound 
image or nuclear spin tomography. To generate a three- 
dimensional image data set for example, the reconstruction 
and assembly of one- or two-dimensional image data which 
have been generated by suitable images is carried out. In 
the case of an ultrasound image, with the aid of an 
ultrasound recording apparatus, two-dimensional ultrasound 
images known as image subdomains are recorded and are so 
arranged in order in a three-dimensional image data set that 
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the two-dimensional image subdomains supply the relative 
arrangement of the image data with the correct orientation 
for the individual three-dimensional image data cubes (known 
as voxels) of the volume data set. 

Such image subdomains generated by two-dimensional recording 
processes normally correspond to a line-by-line scanning 
operation, which records the object line by line in a 
recording direction in which the x-ray machine or e.g. the 
ultrasound transmitter is moved. The image subdomains 
generated in the recording apparatus can be transferred 
digitally or via a video output into a reprocessing 
apparatus or into a data processing system. There, the two- 
dimensional images can be stored or directly reprocessed by 
stacking the individual layers or slices of the object one 
over another or lining them up next to one another, in order 
then to obtain for example a three-dimensional 
representation of the object on the display apparatus. 

If the object is a moving object, the corresponding two- 
dimensional image subdomains can also be provided with an 
associated time datum, so that each image subdomain can be 
allocated to a specific state of motion of the object. By 
means of this time datum, three-dimensional image data sets 
can be generated which also have corresponding time data. 
This is known as a four-dimensional data set. A three- 
dimensional moving object (e.g. the heart of a human) can be 
shown moving on the screen of a data processing apparatus 
(i.e. in four dimensions) by showing the three-dimensional 
image data sets provided with the time data staggered in 
time sequence. 



3 



As soon as a three- or four-dimensional image data set is 
being dealt with, this can be shown for example on the 
screen of a data processing apparatus. In all probability, 
however, the observer of such an image would like to look 
more closely at certain interesting areas of the object, 
e.g. the valve of a heart, the solder point of a 
semiconductor structural element on the printed circuit 
board, or the point of adhesion of a packaging. In this 
case, it is known that the observer, by selecting usually 
two-dimensional planes of section through the three- 
dimensional data set, will analyse the predeterminable 
region by making various planes of section through the 
predeterminable region and will manipulate these 
successively until he obtains the desired section 
represent at ion . 

Such a procedure has the disadvantage, however, that 
"navigation" through the multidimensional data set by means 
of a usually two-dimensional plane of section is time- 
consuming, since the plane has to be adjusted by repeated 
manipulation until the section through the multidimensional 
data set, e.g. through the volume, offers the corresponding 
desired view. For example, such a procedure in the case of 
medical data is often very long-winded and involved, as 
plural stages (of manipulation) are necessary in order to 
"cut clear" the appropriate point of interest, i.e. the 
predeterminable region, and to make it visible. 

It is therefore the object of the invention to indicate a 
method and apparatus for representing a predeterminable 
region which allows the point of interest or predeterminable 
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region within a multidimensional data set to be found and 
represented more quickly and easily. 

The invention achieves this by the characterising features 
of the independent claims 1 and 8. Advantageous embodiments 
and a few examples of the invention are claimed in the 
subclaims and characterised therein. 

The method according to the invention of representing a 
predeterminable region in multidimensional data sets uses in 
a conventional manner a multidimensional image data set, 
which consists of in particular three- or four-dimensional 
image data of an object to be examined. These image data 
can be generated by means of any image of the object e.g. by 
an x-ray image, an ultrasound image, optical and other 
electromagnetic images. After this multidimensional data 
set has been generated, also in a conventional manner, the 
point of interest of the object is represented by making and 
displaying an in particular two-dimensional section through 
the predeterminable region. 

To avoid the aforesaid disadvantages, however, the section 
is not made directly through the multidimensional data set, 
but is defined by at least one vector plane which also lies 
in the multidimensional data set. The at least one vector 
plane is in this case fixed by means of a vector, which is 
in turn fixed in the multidimensional data set or on two- 
dimensional planes of section through the multidimensional 
data set . 

As soon as a two-dimensional plane of section is made 
through the multidimensional data set, for example, in the 
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vicinity of the predeterminable region and the point of 
interest in this two-dimensional plane of section is 
recognised, the vector (called "pointer" below, for ease of 
reference) can be fixed on this point of interest within the 
predeterminable region, so that at least one vector plane 
(called "pointer plane" below for ease of reference) , is 
defined e.g. at the base or the top of the vector or pointer 
plane, and the vector plane or pointer plane then defines 
the section which may be graphically represented and which 
shows the point of interest within the predeterminable 
region or the predeterminable region itself in precisely the 
direction, position and orientation which the user requires. 
The pointer acts as a "navigation tool" . 

Advantageously, the pointer is a directional vector adapted 
to the multidimensional data set with a specifiable 
orientation or a specifiable length which spans the 
predeterminable region. If for example a two-dimensional 
plane of section is made through the solder point of a 
semiconductor element on a printed circuit board and should 
the observer wish to observe a solder error visible on the 
two-dimensional plane of section in a plane which for 
example is not perpendicular to the plane of section, he or 
she can mark by means of the pointer the point recognisable 
on the plane of section and by manipulation of the 
direction, length and orientation of the pointer place the 
pointer virtually "intuitively" in the two-dimensional plane 
of section, in such a manner that the pointer planes fixed 
by the pointer produce precisely the section which the 
observer wants to look at. 
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Advantageously, the pointer is first fixed on a first two- 
dimensional plane of section by determining the orientation 
and length of the pointer and by means of a projection 
positioned perpendicular to a second plane of section, is 
copied on to this second plane of section and there can be 
varied likewise by manipulation of the direction, 
orientation and length. Advantageously, the pointer is in 
this case simultaneously shown in two, three or more planes 
of section, so that a variation of the pointer in one of the 
selected planes of section is also shown on the other planes 
of section. 

Any number of planes of section can be set through the 
multidimensional data set, the pointer being shown as a 
projection in all these planes of section. However, in 
order to achieve an acceleration of the desired section 
through the area of interest (viewing angle) compared to the 
conventional method, it suffices in practice as a preferred 
embodiment of the invention to show the pointer only in one 
or two planes of section and to manipulate it therein. 

By manipulating the pointer, the pointer planes defined by 
the pointer are manipulated simultaneously. For example, 
the at least one pointer plane is fixed by the start point 
or end point of the pointer, the pointer advantageously 
being positioned orthogonally to the pointer plane. 
According to a particular embodiment of the invention, a 
first pointer plane is fixed by the start point and a second 
pointer plane parallel to the first pointer plane by the end 
point of the pointer, so that the pointer is orthogonal to 
both pointer planes . 
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By a length manipulation of the pointer, in this case, the 
distance between the pointer planes is altered. By 
manipulating the orientation of the pointer, the 
corresponding pointer planes are rotated in multidimensional 
space about the centre point or point of rotation of the 
pointer. By manipulating the direction of the pointer, the 
"viewing direction" in multidimensional space is altered, 
i.e. the first pointer plane becomes the second pointer 
plane and vice versa. 

According to a further advantageous embodiment of the 
invention, the direction of the pointer is oriented from the 
start point to the end point of the pointer. This pointer 
also acting as a "navigation arrow" may define an endless 
number of pointer planes which are located between the start 
point and the end point and are each orthogonal to the 
pointer. The representation of the predeterminable region 
or of the point of interest within the multidimensional 
image data set is then carried out by means of the (two- 
dimensional) representation of a section which is defined by 
the pointer planes, which are displaced successively from 
the start point to the end point of the pointer. 

According to a further advantageous embodiment of the 
present invention, the multidimensional data set is cut off 
at the pointer plane or the pointer planes by blanking out 
the corresponding multidimensional image data areas so that 
only those image data are represented multidimensionally 
which lie on one side or the other of the corresponding 
pointer plane or between the pointer planes. If for example 
two pointer planes defined by the pointer are used, the 
three-dimensional image data set for example can be "cut 
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off" on the sides of the pointer planes which lie on the 
side of the respective pointer plane lying opposite to the 
pointer. The observer can then observe the reduced image 
data set from both sides of the pointer planes and thus - 
according to the length of the pointer - observe data set 
areas or areas of the object of different "thicknesses". 

The apparatus according to the invention for representing a 
predeterminable region in multidimensional data sets has 
computing means for generating at least one in particular 
two-dimensional section through the predeterminable region 
and corresponding display means for displaying the section, 
the section being defined by at least one pointer plane 
which lies in the multidimensional data set. As already 
described, this at least one pointer plane is fixed by means 
of a pointer which may be manipulated and fixed by 
manipulation means in the multidimensional data set or on in 
particular two-dimensional planes of section through the 
multidimensional data set. 

Advantageously, the pointer itself may be displayed on a 
display of the multidimensional data set or of the two- 
dimensional planes of section synchronously and may be 
varied in direction or length or orientation by means of the 
manipulation means. For example, while a two-dimensional 
plane of section through the three-dimensional image data 
set of an object is being represented, the pointer can be 
applied to a point of interest in the two-dimensional 
section and so manipulated that this point can be observed 
by the observer in the direction and orientation and at a 
specified point of section which is defined by the pointer 
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plane e.g. at the base of the pointer or at the top of the 
pointer. 

In this case, it is advantageous that during a manipulation 
on a first plane of section, the pointer may be represented 
simultaneously on a second plane of section, and the 
corresponding manipulation is represented synchronously 
there also. Advantageously, the pointer can also be 
manipulated on the second plane of section or further planes 
of section. 

According to a further advantageous embodiment of the 
invention, the pointer is also imaged in the 
multidimensional representation of the data set and can be 
manipulated therein. If for example a three-dimensional 
object is represented in three dimensions, the pointer can 
also be represented and manipulated on this three- 
dimensional image by superposition of a vector data set of 
the pointer (open GL) , known in the art, with the three- 
dimensional data set (two-dimensional representation with 
three-dimensional data) . 

A preferred embodiment of the present invention is explained 
more fully with the aid of the drawings, which show: 

Fig. 1, the three-dimensional data set of a human organ 
(e.g. a heart) generated by means of ultrasound images 
(shown in two dimensions) ; 

Fig. 2, the representation of the data set according to Fig. 
1 by means of two sections S2 and S2 ; 
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Fig. 3, a diagrammatic representation of the vector or 
pointer according to the invention with pointer planes; 

Fig. 4a/4b the arrangement of the vector or pointer with 
vector planes (pointer planes) in the sections SI and S2 
according to Fig. 2; 

Fig. 5, a plan view of a first pointer plane; 

Fig. 6, a plan view of a second pointer plane; 

Fig. 7, a three-dimensional diagram (shown in two 
dimensions) of the image data set (volume) with pointer and 
pointer planes. 

Fig. 1 shows the three-dimensional representation of an 
object such as for example a human heart, wherein the data 
set 2 shown in Fig. 1 has been generated by means of e.g. a 
large number of two-dimensional ultrasound images, which by 
means of conventional reconstruction techniques produce a 
three-dimensional image data set (voxel data set) . This 
volume 1 is generated by corresponding grey-value data of 
the individual image data cubes (voxels) , which are shown in 
Fig. 1 as grey- value data. Fig. 1 furthermore shows a 
predeterminable region 3 which is to be examined in more 
detail . 

Fig. 2 shows two sections SI and S2 through the three- 
dimensional image data set 2 according to Fig. 1, which do 
not necessarily have to be perpendicular to one another. 
Fig. 3 shows diagrammatically a two-dimensional vector or 
pointer 4, which has a specified length, orientation and 



11 

direction, which points from a start point 5 to an end point 
6. Two vector planes (pointer planes) El and E2 are 
positioned orthogonally on the pointer 4, a first pointer 
plane El touching the pointer 4 at the start point 5, whilst 
a second pointer plane E2 touches the pointer 4 at the end 
point 6. 

Figures 4a and 4b show the sections SI (Fig. 4a) and S2 
(Fig. 4b) according to Fig. 2. The predeterminable region 3 
is shown at the upper end of the sections SI and S2 and it 
is assumed that the observer would like to examine or 
observe this predeterminable region more closely. In this 
case, it is supposed that the observer wishes to observe the 
predeterminable region 3 in a certain direction, so that 
then the pointer 4 shown diagrammatical ly in Fig. 3 is 
disposed in the first section SI according to the 
predeterminable region 3 . By varying the length and 
orientation of the pointer 4, automatically the first and 
second pointer planes El and E2 are varied. In Fig. 4a, 
these pointer planes extend orthogonally to the pointer 4 
and perpendicular to the first section SI. 

The two-dimensional vector or pointer 4 shown in Fig. 4a is 
copied by means of a vertical projection on to the second 
section S2 . By copying the pointer 4 on to the second plane 
of section S2, the position of the pointer 4 can be varied 
in any way in three-dimensional space, so that by means of a 
variation of the start point 5 or end point 6 of the pointer 
4, the pointer planes El or E2 can be fixed automatically. 
The lines shown in Fig. 4b represent the projected section 
lines of the second section S2 with the pointer planes El 
and E2 . Obviously, it is also conceivable to fix the 
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pointer 4 first in the second section S2 and then to project 
the same on to the section SI. The sections SI and S2 in 
this case do not necessarily have to be perpendicular to one 
another. 

Figures 5, 6 and 7 show the section S of interest according 
to the selection of the observer. If the observer wants for 
example to observe the predeterminable region 3 in the 
direction of the pointer 4, then he will choose as a section 
S the pointer plane El and be able to observe the 
predeterminable region of interest along the pointer 4 . In 
Fig. 5, the start point 5 is shown, which touches the 
pointer plane El and as it were the "surface" of the section 
S. 

If the observer wants to observe the predeterminable region 
3 in the opposite direction to the pointer 4, he will use as 
the section S the second pointer plane E2 , which is shown in 
Fig. 6. Fig. 6 in this case shows the pointer plane E2, 
i.e. a section made in correct orientation and position 
through the three-dimensional data set 2. 

Fig. 7 shows the overall diagram of the three-dimensional 
volume 1 with the three-dimensional data set 2, which is 
only shown between the pointer planes El and E2 . The 
pointer planes El and E2 are in this case shown in Fig. 7 
parallel to the faces of the volume 1, but the pointer 
planes may lie in any orientation within the volume 1. The 
pointer planes El and E2 are in this case each fixed by the 
start point 5 and the end point 6 of the pointer 4 and 
define the corresponding planes of section through the 
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three-dimensional data set 2. This can be observed 
accordingly from each side of the pointer plane El or E2 . 

According to a particular embodiment of the present 
invention, the pointer 4 can also be manipulated directly in 
the three-dimensional data set 2, as is shown in Fig. 7. 
For example, the start point 5 and/or the end point 6 of the 
pointer on can be varied in position, length, direction 
and/or orientation on the screen of a data processing 
apparatus by "dragging with the mouse", until the observer 
of the three-dimensional data set 2 obtains the correct 
setting. Following this, or even parallel thereto, the 
observer can observe on a further screen or at another point 
of the same screen the section S, which is defined by the 
first pointer plane El and/or the second pointer plane E2 or 
by a further pointer plane, which is set for example through 
the centre of the pointer 4 or at any other point on the 
pointer. 

A method or apparatus of this type is suitable in particular 
for rapidly representing an area of interest of a (diseased) 
human organ, which is represented three-dimensionally by an 
x-ray image or ultrasound image. By moving the pointer 
intuitively, the doctor can immediately define pointer 
planes which permit him to gain the desired view of the 
organ at the predeterminable region 3 . 



